feat: add company self-service public audit pipeline by safal207 · Pull Request #91 · safal207/LiminalQAengineer

safal207 · 2026-07-20T17:59:06Z

Summary

Turns safal207/LiminalQAengineer into a reusable, fail-closed public quality and accessibility audit platform that companies can call from their own GitHub repositories.

A company supplies a committed JSON contract containing its authorized public HTTPS origins and routes. The reusable workflow produces a target × desktop/mobile evidence matrix, screenshots, keyboard/accessibility observations, pinned Lighthouse results, exact-attempt manifests, SHA-256 indexes, reusable outputs, and an aggregate PASS / WARN packet.

Company integration

jobs:
  public-audit:
    uses: safal207/LiminalQAengineer/.github/workflows/company-public-audit.yml@PINNED_LIMINALQA_SHA
    with:
      config_path: .github/liminalqa/public-audit.json
      engine_ref: PINNED_LIMINALQA_SHA
      retention_days: 30
      fail_on: never

The same reviewed SHA or release tag should be used after @ and in engine_ref, so the workflow definition and engine scripts cannot silently come from different revisions.

The workflow also supports workflow_dispatch for forks and approved contracts committed in this repository.

Evidence pipeline

company-owned JSON contract
→ fail-closed validation
→ exact caller SHA + exact engine SHA
→ route × desktop/mobile matrix
→ passive browser observation
→ keyboard/accessibility evidence
→ 1–3 pinned Lighthouse runs
→ per-cell exact-attempt manifests
→ SHA-256 evidence index
→ aggregate PASS/WARN result

Each matrix cell preserves:

HTTP status, final URL, redirect chain and timing;
full-page screenshot;
headings, landmarks, structural counts and public body hash;
sequential focusability and Tab trace;
unnamed accessibility-tree and sequential controls;
nested interactive controls, duplicate IDs, missing alternatives and unlabeled inputs;
sanitized console and failed-request signatures;
raw Lighthouse reports, category scores and core metrics;
exact caller/engine/config/run provenance and file hashes.

Fail-closed safety contract

The schema has no fields for credentials, cookies, custom headers, JavaScript injection, authentication, forms, publishing, direct application APIs, financial actions, fuzzing, exploitation or load testing.

Validation rejects:

non-HTTPS, localhost, local/private/reserved IP and custom-port targets;
credentials embedded in URLs;
origins outside the explicit allowlist;
undeclared query keys;
sensitive-looking query keys such as token, secret, auth, session, password, key, email, phone or account;
any missing or weakened safety boundary;
more than eight targets or unsupported profiles;
unknown schema fields that could smuggle executable instructions.

Permissions remain contents: read. The workflow has no secrets contract and performs no repository writes, external submissions, deployments or merges.

Gate modes

never — evidence only; recommended for onboarding;
high — fail only on aggregate HIGH severity;
any-signal — fail on any aggregate WARN.

Automated quality signals are not a penetration test, vulnerability report, compliance certification, or final root-cause/severity judgment.

Proven end-to-end smoke

A real reusable-workflow smoke against https://example.com/ completed successfully.

workflow run: 29766920933
run attempt: 1
validated engine SHA: b45239e0f284d3205bdc6f3a77de5649bb6fadc5
config SHA-256: 3539e76d19f9a93428242e5d8ed4cb6d5bee2779b303e205420d45b6f836e26f
matrix: 1 target × desktop/mobile = 2/2 accepted manifests
execution: PASS
quality result: WARN / LOW

Passed layers:

pinned actionlint;
Python, Node and JSON syntax;
fail-closed unit tests;
deterministic contract/matrix validation;
reusable prepare with exact caller and engine identities;
desktop browser + keyboard + Lighthouse cell;
mobile browser + keyboard + Lighthouse cell;
exact run/attempt aggregation;
aggregate artifact and outputs;
fail_on: never quality gate.

The example quality result was WARN / LOW only because SEO scored 80 against the example threshold of 85. Both profiles returned HTTP 200, Performance 100, Accessibility 100, Best Practices 96, and zero promoted browser accessibility signals.

Aggregate evidence

artifact ID: 8471326847
artifact: liminalqa-company-audit-29766920933-1
artifact digest: sha256:0f08e1b07a8163f12ef0568651831b6cd1a2e777fdc67ad1ad5b9250e452c70c
result SHA-256: 905ae9bbd3c022e96a4b75bf37c5ce90a9a033db505d320a6bd0aa944e603377
summary SHA-256: 5725b93f8378b683715897b9a065bc8a52e9f0cea400936cd21d07e97bb5c4e2
evidence index SHA-256: 67ea9b2ad6f49eddc0014e6aad240ce8658e49f56a9e05f4471fa69311158950

Exact cell artifacts

desktop artifact: 8471266753
sha256:f907d0e662e3b000670044aef14e9ae5d736a5c5e6cd0ed008ff77232308b171

mobile artifact: 8471247582
sha256:30db3f008a15cc981bb5746f59b927d32db7617a63c81199b4dd1e57ae2255c4

validated contract artifact: 8471215620
sha256:b9b98a1c5a0ba85b74d5bcabdbc9db3675e3fcfb4c5d4dc481e4c6e2c7df0aca

Security Audit on the validated engine head also completed successfully.

Durable smoke evidence

docs/audits/COMPANY_SELF_SERVICE_AUDIT_SMOKE_2026-07-20.md
audits/company-self-service/smoke-result-2026-07-20.json

These preserve run IDs, exact revisions, artifact IDs/digests, result hashes and bounded interpretation after Actions artifacts expire.

Product files

.github/workflows/company-public-audit.yml
.github/workflows/company-public-audit-engine-ci.yml
.github/ISSUE_TEMPLATE/company-public-audit-request.yml
audits/templates/company-public-audit.example.json
scripts/company_public_audit_engine.py
scripts/company_public_browser_probe.mjs
tests/test_company_public_audit_engine.py
docs/COMPANY_SELF_SERVICE_AUDIT.md
docs/examples/company-audit-caller.yml

Evidence files

docs/audits/COMPANY_SELF_SERVICE_AUDIT_SMOKE_2026-07-20.md
audits/company-self-service/smoke-result-2026-07-20.json

Authority

Evidence and optional quality gates only. No ownership, approval, external submission, deployment or merge authority is granted. This PR remains draft and unmerged pending release/tagging decisions.

coderabbitai · 2026-07-20T17:59:16Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: cdc1f78e-967c-4a8d-ad60-8ebbf167a92a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch agent/company-self-service-audit-v0-1

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

safal207 · 2026-07-20T19:08:18Z

Multi-company hardening discovered during real use

The OpenAI Codex / Claude Code / Cursor portfolio exposed two reusable-workflow namespace constraints that were not visible in the single-company example.com smoke:

concurrency.group currently keys only on repository/ref, so multiple reusable calls with different config_path values under the same caller ref share one group. GitHub retains one running and one pending run; newer pending company calls can replace earlier pending calls even with cancel-in-progress: false.
Contract, cell, and aggregate artifact names are based on run/attempt/cell but not contract/company identity. Multiple company calls inside one top-level run would collide on names such as liminalqa-contract-<run>-<attempt> and ...-product-desktop.

No product conclusion was derived from canceled attempts. Final company evidence was rerun under isolated child PR refs (#93, #94, #95), producing three independent 10/10 exact-attempt sets.

Recommended hardening before multi-company release:

contract_id = sha256(canonical validated config)[0:12]

concurrency:
  group: liminalqa-company-audit-${repository}-${ref}-${contract_id}

artifacts:
  liminalqa-contract-${run_id}-${attempt}-${contract_id}
  liminalqa-cell-${run_id}-${attempt}-${contract_id}-${cell_id}
  liminalqa-company-audit-${run_id}-${attempt}-${contract_id}

The aggregate must continue downloading only the active contract identity and enforcing exact run/attempt/config SHA. This is a platform hardening item, not an audited-company defect.

safal207 added 10 commits July 20, 2026 20:53

feat: add company public audit contract example

01bfa9e

feat: add fail-closed company audit engine

467e432

feat: add generic passive company browser probe

7789651

feat: add reusable company public audit pipeline

2169ac5

fix: harden reusable company audit workflow

2f37cd9

test: cover company audit fail-closed contract

95a66e1

ci: validate company self-service audit engine

f6eb009

docs: add reusable audit caller example

290ce60

docs: document company self-service audit pipeline

d7fb3ae

feat: add company audit onboarding form

bfc3745

ci: validate reusable company audit workflow syntax

928c849

safal207 closed this Jul 20, 2026

safal207 reopened this Jul 20, 2026

safal207 added 9 commits July 20, 2026 21:05

docs: remove onboarding label dependency

ea53a55

ci: diagnose company audit workflow registration

3a33525

ci: isolate reusable workflow registration

dc04423

ci: isolate reusable workflow parser failure

2401f5b

ci: restore company audit workflows after registration check

0c77868

ci: preserve actionlint diagnostics

8074975

ci: scope actionlint shellcheck exceptions

b45239e

docs: preserve company audit smoke evidence

0bb3860

docs: add company audit smoke report

3e55bcf

safal207 mentioned this pull request Jul 20, 2026

audit: preserve OpenAI Codex, Claude Code, and Cursor public evidence #92

Draft

Navigation Menu

feat: add company self-service public audit pipeline - #91

feat: add company self-service public audit pipeline#91
safal207 wants to merge 20 commits into
mainfrom
agent/company-self-service-audit-v0-1

safal207 commented Jul 20, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jul 20, 2026 •

edited

Loading

Review skipped

Uh oh!

safal207 commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

safal207 commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Company integration

Evidence pipeline

Fail-closed safety contract

Gate modes

Proven end-to-end smoke

Aggregate evidence

Exact cell artifacts

Durable smoke evidence

Product files

Evidence files

Authority

Uh oh!

coderabbitai Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

safal207 commented Jul 20, 2026

Multi-company hardening discovered during real use

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

safal207 commented Jul 20, 2026 •

edited

Loading

coderabbitai Bot commented Jul 20, 2026 •

edited

Loading